Not Applicable
Reference to a xe2x80x9cMicrofiche Appendixxe2x80x9d
Not Applicable
1. Field of the Invention
The present invention relates to modeling the function of a neural synapse in hardware thus allowing for building of neural networks or alike learning and adaptive systems using low-power conventional MOSFETs integrated on-chip with other analog or digital application circuits.
2. Description of the Related and Prior Art
In spite of the immense advance in the development of the modem VLSI technology, digital circuits or systems reach the critical point from its computational speed and integration. In implementing real-time digital signal processing applications, the conventional VLSI technology has many limitations with regard to chip area and its speed, due to complexity of both hardware and software processing algorithms. On the other hand, artificial neural networks offer great potential in many applications. This poses the need of a simple, fast and efficient solution for implementation of such neural networks in a conventional CMOS fabrication technology along with the rest of the integrated circuit application. Such characteristics can be most often satisfied by the utilization of simple analog signal processing elements, in particular MOSFETs.
Prior efforts for single-MOSFET synapse circuits largely include use of floating gate MOSFET devices (for example: U.S. Pat. No. 6,023,422; U.S. Pat. No. 5,986,927; U.S. Pat. No. 5,914,894; U.S. Pat. No. 5,864,242; U.S. Pat. No. 5,825,063; U.S. Pat. No. 5,818,081; U.S. Pat. No. 5,627,392; U.S. Pat. No. 5,621,336; U.S. Pat. No. 5,457,771; U.S. Pat. No. 5,336,936; U.S. Pat. No. 5,331,215; U.S. Pat. No. 5,253,196; U.S. Pat. No. 5,204,549; U.S. Pat. No. 5,027,171) which utilize various structures and methods to store/remove electrical charges on/from a floating gate or node through insulator by tunneling (Fowler-Nordheim) or hot-electron-injection. Beside the many advantages that those structures offer, they rely on conditions that are hard to meet in a conventional CMOS fabrication process. For example, at the voltages and oxide thicknesses used in conventional silicon MOS processing, the tunneling probability is exceedingly small, likewise hot-electron-injection requires much higher electric fields and supply voltages than normal. The latter is in direct conflict with the tendency of lowering supply voltages with the constant decrease of the device geometries and increase of the scale of integration.
Other related prior art includes analog multipliers build by coupled current mirrors (U.S. Pat. No. 5,914,868), four-quadrant square-law circuits (U.S. Pat. No. 5,864,255) or pair of MOSFETs and a symmetrical voltage source (U.S. Pat. No. 5,254,889) all capable of removing non-linear current component in one form or another to improve multiplier linearity. Some solutions utilize voltage-current conversion circuits employing differential pair of MOS transistors (U.S. Pat. No. 5,704,014) or current controlled current conveyor circuits (U.S. Pat. No. 5,206,541). There are also solutions to use multiple-bit-split MOSFET gates (U.S. Pat. No. 5,442,209). It should be mentioned that this list is not in any way comprehensive or inclusive and does not represent the complete scope and class of search for prior art performed by the inventor. For sake of briefness, herein will not be mentioned quite innovative solutions for building artificial neural networks as, for example, xe2x80x9cInformation processing device capable of optically writing synapse strength matrixxe2x80x9d (U.S. Pat. No. 5,546,504) in which a matrix of plurality of neurons is realized through a combination of a light-emitting and light-receiving functions of two overlapping molecular films. The above-related art is mentioned in order to point out that most of the solutions require special integrated circuit technology, show performance dependencies as to the quality or the technology parameters of the used fabrication process or require special ambient (for UV floating-gate programming for example). Therefore, in general practice, those solutions do not allow for easy on-chip integration with other integrated circuit applications implemented on most widely used conventional CMOS processes.
Naturally, it is desired to provide a simple single-transistor synapse for modeling neural synaptic connection function and for building artificial neural networks in VLSI integrated circuits, which offers following advantages and features:
1. use of a single, conventional MOSFET to model a neural synapse function;
2. low power consumption;
3. independence of the model quality with regard to quality of the CMOS fabrication process;
4. simplicity and feasibility to dense integration;
5. preferred conventional CMOS implementation which allows for integration with conventional analog and digital IC applications;
6. current signal input and output allowing for straightforward and easy multi-layer neural network interconnection, simple synapse-output summing, avoidance of parasitic capacitive effects on both input and output transient signals therefore allowing for faster processing in real-time applications;
This and other features of the invention will be apparent to a person of ordinary skill in the art from the description of the invention contained herein.
Specification description of the claimed invention and disclosure will be facilitated by the definition of the meaning of the following terms. Term definitions given herein are intended to help in limiting redundancy and unduly multiplication of terms in the disclosure and should be used to better comprehend the spirit and scope of the invention. The definitions given are not intended to limit and define the scope of the appended claims rather than to provide antecedent support of a claim in cases where such support can be more easily found by the interpretation of the terms given herein and used in the disclosure, or in cases where these definitions help avoid inconsistencies between claim and specification disclosure or prior art, should such arise.
Neural synapsexe2x80x94although xe2x80x98neuralxe2x80x99 refers to a biological synaptic connection formed between a dendrite tree and axon terminal of a neuron (most often cerebral pyramidal cell), herein it should be most often interpreted as artificial synapse model, mathematical or functional abstract of its biological counterpart. Synapse, neural synapse or synaptic connection will most often be used interchangeably in the application.
Synapse strength or weightxe2x80x94the quantity that can be used to show the degree of proportion between the quantity used to represent pre-synaptic activity (input stimulus) and the quantity representing post-synaptic activity (weighted input stimulus).
Neural synapse functionxe2x80x94although biological synapse function comprises functions of excitation or inhibition to effectively strengthen or weaken the transmission of a pre-synaptic signal to a post-synaptic signal and also learning functions (to keep or adjust its strength with respect to synaptic activity), herein, a synapse function should be interpreted as the multiplication function between the pre-synaptic signal quantity and synapse strength (or weight) quantity employed in the computational model most often used to describe the synapse.
Neuronxe2x80x94most often should be interpreted as functional abstract model of its biological counterpart. The latter model comprises of a linear combiner producing a sum of weighted input stimuli, further processed by a non-linear activation function, of sigmoidal or hard-limiting type, applied to limit the amplitude of the neuron output activity potential.
Neural networkxe2x80x94a collection of interconnected neurons where a neuron input is either connected to previous layer neuron""s output or receives input stimulus from an input node. Since the type or actual connectivity structure is immaterial to the present invention no further definition of neural network types and connectivity patterns will be given herein.
It is an object of the present invention to provide a method for modeling a neural synapse in hardware, using VLSI technology. More specifically, a method utilizing a single conventional MOSFET to model a neural synapse by effectively computing the product of an input signal (pre-synaptic activity) represented by gate-source voltage of latter MOSFET with synapse strength (or weight) represented by the drain-source voltage of same MOSFET.
Another object of the present invention is to provide a high-performance neural network synapse circuit operating in low voltage level range as a particular embodiment, allowing for variable weight setting (xcexdDS voltage) while keeping the latter independents within certain operational range, from the synapse output (post-synaptic) signal (iD current).
In accordance with an aspect of the present invention, there is provided as prior art a first-order approximation of large-signal, low-frequency computational model describing a MOSFET in non-saturation region of operation and, as part of the object of the invention described herein, is presented the use of the above to model the multiplication function of a synapse. It should be noted that the presented model considers DC operating mode of operation or operation at analog signal frequencies as low as to keep the validity of relationships shown and practically render frequency dependent terms or effects insignificant. However, this does not limit the applicability of the model since in most cases this condition can be satisfied during consecutive short time periods of processing intervals.
In accordance with an aspect of the present invention, there is provided a neural network synapse comprising: a single conventional MOSFET with applied gate-source voltage as input signal, applied drain-source voltage as weight control signal; a preferred embodiment of an artificial neuron comprising a: synapse current input; synapse output current summing node; weight control circuit; neuron response current output.